Video coding device

ABSTRACT

The prediction error value calculated by a motion estimating portion is used for selecting the frame or field orthogonal transform coding mode. The frame prediction error value EFr and the field prediction error value EFi are calculated by the motion estimating portion and entered into a comparator of an orthogonal transform mode selecting portion which in turn compares them with each other and generates an orthogonal transform mode selecting signal TC. The use of the prediction error values for selecting the orthogonal transform mode makes it possible to reduce the size of the necessary hardware and saves much time of data processing necessary for judgment of the orthogonal transform mode selection.

BACKGROUND OF THE INVENTION

The present invention relates to a video coding device and moreparticularly to a video coding device which is capable of selectingeither of two suitable orthogonal transform modes, i.e., frame-by-frameor field-by-field orthogonal transform coding modes according to adetermination based on motion predictive information, and which,thereby, realizes a simplified structure of its hardware without anyimpairment of image quality.

A frame-and-field adaptive interframe video-coding method is known,which can selectively use frame-by-frame motion prediction orfield-by-field motion prediction, intraframe coding or interframecoding, frame-by-frame orthogonal transform or field-by-field orthogonaltransform for digital transmitting or recording video informationextracted by interlaced scanning.

The general frame-and-field adaptive type interframe video-coding devicewhich comprises an input terminal, a subtracter, a intraframe-interframecoding-mode selecting portion, a change-over switch, a frame-to-fieldconverter, an orthogonal transform mode selecting portion, a change-overswitch, an orthogonal transform coder, a multiplexor (MUX), an outputterminal, an orthogonal transform decoder, a field-to-frame converter, achange-over switch, an adder, a change-over switch, a frame memory, apredictor and a motion estimating portion.

A frame to be coded is divided into macro-blocks each consisting of2N×2N pixels and inputted into the coding device through the inputterminal. These inputted macro-blocks are indicated by X. A macro-blockX has an interlaced structure, i.e., odd-numbered lines correspond topixels of odd-numbered fields in the macro-block X and even-numberedlines correspond to pixels of even-numbered fields in the macro-block X.

By using the macro-block X and a reference frame stored in the framememory, the motion estimating portion calculates a motion vector MV and,at the same time, decides which of the frame-by-frame and thefield-by-field prediction modes is suitable and produces a predictionmode selecting signal PC. Hereinafter, the prediction to be made perframe is called frame prediction mode and the prediction to be made perfield is called field prediction mode. The number of motion vectors isone per macro-block for frame prediction mode and two per macro-block(one vector for an odd-numbered field and one vector for aneven-numbered field) for field prediction mode. For bidirectionalprediction referring to preceding and succeeding reference frames, thenumber of vectors is two times the above-mentioned corresponding number.

The predictor calculates a predictive macro-block P according to amotion vector MV and a prediction mode selecting signal PC. Namely, aframe predictor generates a frame-predictive macro-block PFr and a fieldpredictor generates a field-predictive macro-block PFi. A change-overswitch selects one of macro-blocks PFr and PFi according to theprediction mode selecting signal PC. The selected macro-block is denotedby P.

The subtracter determines a difference between macro-blocks X and P toobtain a interframe differential macro-block E. Theintraframe-interframe coding-mode selecting portion compares themacro-block X with the macro-block E, decides which macro-block X or Eis to be encoded and generates a coding-mode selecting signal EC tooperate the change-over switch. The intraframe coding mode is appliedwhen X is selected and the interframe coding mode is applied when E isselected. The macro-block selected by the change-over switch isdesignated by Fr. This macro-block Fr has an interlaced structure.

The macro-block Fr is transferred to the frame-to-field converterwherein it is subjected to frame-to-field conversion. The rearrangedmacro-block is denoted by Fi. The upper half of the macro-block Fi iscomposed of 2N×N pixels of the odd-numbered fields and the lower half iscomposed of 2N×N pixels of the even-numbered fields.

The orthogonal transform-mode selecting portion judges which macro-blockFr or Fi is to be encoded by orthogonal transform coding method,generates an orthogonal transform-mode selecting signal TC to effect thechange-over switch to select either one of macro-blocks Fr and Fi. Theselected block is denoted by B. The orthogonal transform coder encodesthe selected macro-block B of 2N×2N pixels by orthogonal transformationin 4 blocks (upper left, upper right, lower left and lower right), eachhaving N×N pixels. The frame orthogonal transform coding mode is appliedwhen Fr is selected and the field orthogonal transform coding mode isapplied when Fi is selected.

An output Y of the orthogonal transform coder is multiplexed with themode signals TC, EC and PC and a motion vector MV by the multiplexorwhich generates, at its output terminal, a multiplexed output Z to betransmitted or recorded. To obtain an image reproducible at a decodingside, a coding side performs the decoding operation simultaneously withthe coding operation and stores a decoded macro-block in the framememory. The encoded data Y enters into the orthogonal transform decoderwhich decodes the data by orthogonal transformation and obtains adecoded macro-block Fr'.

The field-to-frame converter converts the macro-block Fr' into framearrangement by reversing the procedure of the frame-to-field conversion.The converted macro-block is denoted by Fi'. Either of the macro-blocksFr' and Fi' is selected by the change-over switch according to theorthogonal transform mode selecting signal TC. The macro-block Fr' isselected for the frame orthogonal transform mode and the macro-block Fi'is selected for the field orthogonal transform mode.

The adder adds a selected macro-block X' to a predictive macro-block Pto obtain a resulting macro-block E'. Either of the macro-blocks X' andE' is selected by the change-over switch according to theintraframe-interframe coding mode selecting signal EC. The macro-blockX' is selected for the intraframe coding mode and the macro-block E' isselected for the interframe coding mode. A decoded macro-block D' isthus obtained. On completion of processing one frame, there is obtaineda complete decoded frame which will be used as a reference frame forcoding a further frame.

To assure adaptability for decoding starting from the halfway point ofthe sequence, the above-mentioned coding method usually periodicallyinserts a frame whose macro-blocks are all intraframe-coded in theintraframe coding mode. Any frame other than the intraframe-coded frameis hereinafter called an interframe-prediction coded frame. A firstmethod for changing-over the frame orthogonal transform mode to thefield orthogonal transform mode and vice versa is "MPEG2 interframeprediction method" which is disclosed in Technical Review of the TVConference, Vol. 16, No. 61, pp. 37-42. The summary of this method is asfollows: A maximum frequency power (in sense of Hadamard's transform) inthe vertical direction of the macro-block Fr and a total of maximumfrequency power values in vertical direction of two fields (an upperhalf and a lower half of the macro-block Fi) are compared with eachother. The field-transform mode is selected when the former is largerthan the latter, whereas the frame-transform mode selected when thelatter is larger than the former.

An orthogonal transform mode selecting portion according to theabove-mentioned method has two vertical maximum frequency powercalculators and a comparator.

The macro-blocks Fr and Fi are transferred to the vertical maximalfrequency power calculators respectively. The power values calculated bythe calculators for the macro-blocks Fr and Fi are compared with eachother by the comparator to decide a transform mode TC to be selected.The vertical maximum frequency power has the following expression (1):##EQU1## where x(1≦x≦2N) is the abscissa of a pixel in a macro-block,y(1≦y≦2N) is the ordinate of a pixel in a macro-block and O(x,y) is apixel value of the coordinates in a macro-block.

The vertical maximal frequency power calculating portion includeschange-over switches, a delay circuit, a subtracter, a multiplier, anadder, a register, a latch and a control portion.

The change-over switches operate, respectively, once a line of theblock. When pixels of an odd-numbered line enters they are switched tothe one side, and when pixels of an even-numbered line enters they areswitched to the other side. The delay circuit can delay the input by 2Npixels. Accordingly, O(x,y) and O(x,y-1) are inputted to the subtracter.The output "O(x,y)-O(x,y-1)" of the subtracter is squared by themultiplier and accumulatively added to the preceding sum by the adderand the register.

The addition according to the equation (1) is carried out only when y isan even number. Accordingly, if y is an odd number the input of themultiplier is held at 0 by the action of the change-over switch not tochange the accumulated value. The latch is controlled to maintain aresult of accumulation for one block only. On completion of accumulationof one block the register is reset for calculation of a succeedingblock. The control portion controls a series of the above-mentionedoperations.

Japanese Laid-open Patent Publication No. 5-91500 discloses a highlyefficient video CODEC device which uses the second method forchanging-over from the frame orthogonal transform mode to the fieldorthogonal transform mode and vice versa, which will be described below:

Differential values EFD and EFM, which are defined by the followingequation, are calculated for a macro-block Fr. The field orthogonaltransform mode is applied if a difference between differential valuesEFM and EFD exceeds a certain threshold (i.e., EFM-EFD>T2), and theframe orthogonal transform mode is applied in the other case than theabove-mentioned. ##EQU2##

The equations (2) and (3) correspond to the equation (1) whose squaredvalue is substituted by absolute value for Fi and Fr respectively.

The methods use data on macro-block pixels to discriminate which mode ofthe frame orthogonal transform mode and field orthogonal transform modeis to be conducted.

The first method requires a hardware for calculating a verticallymaximal frequency power for selecting the frame orthogonal transformmode or the field orthogonal transform mode. This hardware includes atleast a multiplier, two adders, a memory for delay line and a controlcircuit and is, therefore, of considerably large size. In addition, themaximal frequency power calculation must be conducted for a framemacro-block (Fr) and a field macro-block (Fi), which requires thedoubled scale of the hardware.

On the other hand, the second method has no need of multipliers.Therefore, the hardware necessary to effect the method iscorrespondingly reduced in comparison with the first method but stilllarge in scale. Both the first and second methods use data on all pixelsin each macro-block to judge which one of the frame mode and field modeto select. As a result, both methods must treat with a large number ofinput data and perform much time-consumable operations for dataprocessing.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a video codingdevice which is capable of adaptively selecting frame or fieldorthogonal transform mode according to a determination based on a motionprediction information, assuring prevention of impairment of imagequality without enlargement of its hardware.

It is another object of the present invention to provide a video codingdevice which is capable of performing the interframe video coding withadaptively selecting either frame or field orthogonal transform mode byusing a prediction error value of the frame predictive mode, aprediction error value of the field predictive mode and motion vectors,and which is featured by the prediction error values calculatedrespectively in the frame prediction mode and in the field predictionmode which are compared with each other to determine which one is largerthan the other and the orthogonal transform mode is selected.

It is another object of the present invention to provide a video codingdevice which is capable of dividing each frame of a video sequenceobtained by interlaced scanning into codable blocks and generatingpredicted image blocks from a preceding-frame and a succeeding frame byusing motion vectors determined by estimation, the estimation beingperformed in one of selective modes comprising a frame prediction modeto predict by using a motion vector per codable block and a fieldprediction mode to predict by using motion vectors per respectiveodd-numbered field consisting of odd-numbered lines and even-numberedfield consisting of even-numbered lines, comprising a motion estimatingportion for determining an error value of frame mode prediction and anerror value of field mode prediction and an orthogonal transform modeselecting portion for adaptively selecting a field orthogonal transformmode when the error value of the frame-mode prediction is larger thanthe error value of the field mode prediction and selecting a frameorthogonal transform mode when the error value of the frame modeprediction is smaller than an error value of the field mode prediction.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a conventional frame-field adaptableinterframe-coding device.

FIG. 2 is a block diagram of an ordinary predicting unit.

FIG. 3 shows a conventional frame-field converting process.

FIG. 4 is a block diagram of an ordinary orthogonal transform modeselecting portion shown in FIG. 1.

FIG. 5 is a block diagram of a maximal frequency power calculatingportion shown in FIG. 4.

FIG. 6 is a block diagram for explaining a video coding device embodyingthe present invention.

FIG. 7 is a block diagram showing an embodiment of an orthogonaltransform-mode selecting portion of the device shown in FIG. 6.

FIG. 8 shows output characteristics of an comparator of the device shownin FIG. 7.

FIG. 9 is a block diagram showing another embodiment of an orthogonaltransformation-mode selecting portion of the device shown in FIG. 6.

FIG. 10 is a block diagram showing an embodiment of a motion-valuecalculating portion shown in FIG. 9.

FIG. 11 is a block diagram showing another embodiment of a motion valuecalculating portion shown in FIG. 9.

FIG. 12 is a block diagram showing an embodiment of a motion-valuecalculator shown in FIG. 11.

FIG. 13 is a block diagram showing another embodiment of a motion-valuecalculator shown in FIG. 11.

FIG. 14 shows an example of timing chart of data and control signal inthe calculator shown in FIG. 13.

FIG. 15 is a block diagram showing a further embodiment of amotion-value calculator shown in FIG. 11.

FIG. 16 is a block diagram showing another embodiment of a motion-valuecalculator shown in FIG. 11.

FIG. 17 is a block diagram for explaining another embodiment of a videocoding device according to the present invention.

PREFERRED EMBODIMENTS OF THE INVENTION

FIG. 1 is a block diagram showing a general frame-and-field adaptivetype interframe video-coding device which comprises an input terminal 1,a subtracter 2, a intraframe-interframe coding-mode selecting portion 3,a change-over switch 4, a frame-to-field converter 5, an orthogonaltransform mode selecting portion 6, a change-over switch 7, anorthogonal transform coder 8, a multiplexor (MUX) 9, an output terminal10, an orthogonal transform decoder 11, a field-to-frame converter 12, achange-over switch 13, an adder 14, a change-over switch 15, a framememory 18, a predictor 17 and a motion estimating portion 18.

A frame to be coded is divided into macro-blocks each consisting of2N×2N pixels and inputted into the coding device through the inputterminal 1. These inputted macro-blocks are indicated by X. Amacro-block X has an interlaced structure, i.e., odd-numbered linescorrespond to pixels of odd-numbered fields in the macro-block X andeven-numbered lines correspond to pixels of even-numbered fields in themacro-block X.

By using the macro-block X and a reference frame stored in the framememory 16, the motion estimating portion 18 calculates a motion vectorMV and, at the same time, decides which of the frame-by-frame and thefield-by-field prediction modes is suitable and produces a predictionmode selecting signal PC. Hereinafter, the prediction to be made perframe is called frame prediction mode and the prediction to be made perfield is called field prediction mode. The number of motion vectors isone per macro-block for frame prediction mode and two per macro-block(one vector for an odd-numbered field and one vector for aneven-numbered field) for field prediction mode. For bidirectionalprediction referring to preceding and succeeding reference frames, thenumber of vectors is two times the above-mentioned corresponding number.

FIG. 2 is a block diagram showing a general configuration of thepredictor included in the device of FIG. 1. Numeral 19 indicates a framepredictor, numeral 20 a field predictor and numeral 21 a change-overswitch. The predictor 17 calculates a predictive macro-block P accordingto a motion vector MV and a prediction mode selecting signal PC. Namely,the frame predictor 19 generates a frame-predictive macro-block PFr andthe field predictor 20 generates a field-predictive macro-block PFi. Thechange-over switch 21 selects one of macro-blocks PFr and PFi accordingto the prediction mode selecting signal PC. The selected macro-block isdenoted by P.

The subtracter 2 determines a difference between macro-blocks X and P toobtain a interframe differential macro-block E. Theintraframe-interframe coding-mode selecting portion 3 compares themacro-block X with the macro-block E, decides which macro-block X or Eis to be encoded and generates a coding-mode selecting signal to operatethe change-over switch 4. The intraframe coding mode is applied when Xis selected and the interframe mode is applied when E is selected. Themacro-block selected by the change-over switch is designated by Ft. Thismacro-block Fr has an interlaced structure.

The macro-block Fr is transferred to the frame-to-field converter 5wherein it is subjected to frame-to-field conversion, i.e., it isrearranged into lines as shown in FIG. 3. The rearranged macro-block isdenoted by Fi. The upper half of the macro-block Fi is composed of 2N×Npixels of the odd-numbered fields and the lower half is composed of 2N×Npixels of the even-numbered fields.

The orthogonal transform-mode selecting portion 6 judges whichmacro-block Fr or Fi is to be encoded by orthogonal transfer codingmethod, generates an orthogonal transform-mode selecting signal TC toeffect the change-over switch 7 to select either one of macro-blocks Frand Fi. The selected block is denoted by B. The orthogonal transformcoder 8 encodes the selected macro-block B of 2N×2N pixels by orthogonaltransformation in 4 blocks (upper left, upper right, lower left andlower right), each having N×N pixels. The frame orthogonal transformcoding mode is applied when Fr is selected and the field orthogonaltransform coding mode is applied when Fi is selected.

An output Y of the orthogonal transform coder 8 is multiplexed with themode signals TC, EC and PC and a motion vector MV by the multiplexor 9which generates, at its output terminal 10, a multiplexed output Z to betransmitted or recorded. To obtain an image reproducible at a decodingside, a coding side performs the decoding operation simultaneously withthe coding operation and stores a decoded macro-block in the framememory 16. The encoded data Y enters into the orthogonal transformdecoder 11 which decodes the data by orthogonal transformation andobtains a decoded macro-block Fr'.

The field-to-frame converter 12 converts the macro-block Fr' into framearrangement by reversing the procedure of the frame-to-field conversion.The converted macro-block is denoted by Fi'. Either of the macro-blocksFr' and Fi' is selected by the change-over switch 13 according to theorthogonal transform mode selecting signal TC. The macro-block Fr' isselected for the frame orthogonal transform mode and the macro-block Fi'is selected for the field orthogonal transform mode.

The adder 14 adds a selected macro-block X' to a predictive macro-blockP to obtain a resulting macro-block E'. Either of the macro-blocks X'and E' is selected by the change-over switch 15 according to theintraframe-interframe coding mode selecting signal EC. The macro-blockX' is selected for the intraframe coding mode and the macro-block E' isselected for the interframe coding mode. A decoded macro-block D' isthus obtained. On completion of processing one frame, there is obtaineda complete decoded frame which will be used as a reference frame forcoding a further frame.

To assure adaptability for decoding starting from the halfway point ofthe sequence, the above-mentioned coding method usually periodicallyinserts a frame whose macro-blocks are all intraframe-coded in theintraframe coding mode. Any frame other than the intraframe-coded frameis hereinafter called an interframe-prediction coded frame. A firstmethod for changing-over the frame orthogonal transform mode to thefield orthogonal transform mode and vice versa is "MPEG2 interframeprediction method" which is disclosed in Technical Review of the TVConference, Vol. 16, No. 61, pp. 37-42. The summary of this method asfollows: A maximum frequency power (in sense of Hadamard's transform) inthe vertical direction of the macro-block Fr and a total of maximumfrequency power values in vertical direction of two fields (an upperhalf and a lower half of the macro-block Fi) are compared with eachother. The field-transform mode is selected when the former is largerthan the latter, whereas the frame-transform mode selected when thelatter is larger than the former.

FIG. 4 is a block diagram of an orthogonal transform mode selectingportion according to the above-mentioned method. In FIG. 4, numerals 22aand 22b designate vertical maximum frequency power calculators andnumeral 23 designates a comparator.

The macro-blocks Fr and Fi are transferred to the vertical maximalfrequency power calculators 22a and 22b respectively. The power valuescalculated by the calculators 22a and 22b for the macro-blocks Fr and Fiare compared with each other by the comparator 23 to decide a transformmode TC to be selected. The vertical maximum frequency power has thefollowing expression (1): ##EQU3## where x(1≦x≦2N) is the abscissa of apixel in a macro-block, y(1≦y≦2N) is the ordinate of a pixel in amacro-block and O(x,y) is a pixel value of the coordinates in amacro-block.

FIG. 5 is a block diagram of the vertical maximal frequency powercalculating portion shown in FIG. 4, which includes change-over switches24, 27, a delay circuit 25, a subtracter 26, a multiplier 28, an adder29, a register 30, a latch 31 and a control portion 32. The change-overswitches 24 and 27 operate, respectively, once a line of the block. Whena pixel of an odd-numbered line enters they are switched to the side A,and when a pixel of an even-numbered line enters they are switched tothe side B. The delay circuit 25 can delay the input by 2N pixels.Accordingly, O(x,y) and O(x,y-1) are inputted to the subtracter 26. Theoutput "O(x,y)-O(x,y-1)" of the subtracter 26 is squared by themultiplier 28 and accumulatively added to the preceding sum by the adder29 and the register 30.

The addition according to the equation (1) is carried out only when y isan even number. Accordingly, if y is an odd number the input of themultiplier 28 is held at 0 by the action of the change-over switch 27not to change the accumulated value. The latch 31 is controlled tomaintain a result of accumulation for one block only. On completion ofaccumulation of one block the register 30 is reset for calculation of asucceeding block. The control portion 32 controls a series of theabove-mentioned operations.

Japanese Laid-open Patent Publication No. 5-91500 discloses a highlyefficient video CODEC device which uses the second method forchanging-over from the frame orthogonal transform mode to the fieldorthogonal transform mode and vice versa, which will be described below:

Differential values EFD and EFM, which are defined by the followingequation, are calculated for a macro-block Fr. The field orthogonaltransform mode is applied if a difference between differential valuesEFM and EFD exceeds a certain threshold (i.e., EFM-EFD>T2), and theframe orthogonal transform mode is applied in the other case than theabove-mentioned. ##EQU4##

The equations (2) and (3) correspond to the equation (1) whose squaredvalue is substituted by absolute value for Fi and Fr respectively.

The methods use data on macro-block pixels to discriminate which mode ofthe frame orthogonal transform mode and field orthogonal transform modeis to be conducted.

The first method requires hardware for calculating a vertically maximalfrequency power for selecting the frame orthogonal transform mode or thefield orthogonal transform mode. This hardware includes at least amultiplier, two adders, a memory for delay line and a control circuitand is, therefore, of considerably large size. In addition, the maximalfrequency power calculation must be conducted for a frame macro-block(Fr) and a field macro-block (Fi), which requires the doubled scale ofthe hardware.

On the other hand, the second method has no need of adders. Therefore,the hardware necessary to effect the method is correspondingly reducedin comparison with the first method but still large in scale. Both thefirst and second methods use data on all pixels in each macro-block tojudge which one of the frame mode and field mode to select. As theresult, both methods must treat with a large number of input data andperform much time-consumable operations for data processing.

In view of the foregoing, the present invention was made to provide avideo coding device which is capable of adaptively selecting frame orfield orthogonal transform mode according to the judgment based on amotion prediction information, assuring prevention of impairment ofimage quality without enlargement of its hardware.

To solve the above-mentioned problems, the present invention provides:

(1) a video coding device capable of dividing each frame of a videosequence obtained by interlaced scanning into codable blocks andgenerating predicted image blocks from a preceding-frame and asucceeding frame by using motion vectors determined by estimation, saidestimation being performed in one of selective modes comprising a frameprediction mode to predict by using a motion vector per codable blockand a field prediction mode to predict by using motion vectors perrespective odd-numbered field consisting of odd-numbered lines andeven-numbered field consisting of even-numbered lines, comprising amotion estimating portion for determining an error value of frame modeprediction and an error value of field mode prediction and an orthogonaltransform mode selecting portion for adaptively selecting a fieldorthogonal transform mode when the error value of the frame-modeprediction is larger than the error value of the field mode predictionand selecting a frame orthogonal transform mode when the error value ofthe frame mode prediction is smaller than an error value of the fieldmode prediction;

(2) a video coding device as defined in item (1), characterized in thatthe orthogonal transform mode selecting portion has comparing means tocompare an error value of the frame mode prediction with an error valueof the field mode prediction error;

(3) a video coding device as defined in item (1), characterized in thatthe orthogonal transform mode selecting portion has calculating meansfor calculating a motion value from a motion vector determined forestimating the motion and comparing means for comparing the calculatedmotion value with a given value, and selects a field orthogonaltransform mode when the motion value is larger than the given value andselects a frame orthogonal transform mode when the motion value issmaller than the given value;

(4) a video coding device as defined in item (3), characterized in thatthe maximum motion value corresponding to respective motion vectorscalculated for one macro block is selected as the motion value of saidmacro block;

(5) a video coding device as defined in item (3), characterized in thata total of motion values corresponding to respective motion vectorscalculated for one macro block is selected as the motion value of saidmacro block;

(6) a video coding device as defined in any of items (3) to (5),characterized in that a total of absolute values of vertical componentsand a horizontal components of motion vectors is selected as the motionvalue;

(7) a video coding device as defined in any of items (3) to (5),characterized in that a total of absolute values of vertical componentsand a horizontal components of motion vectors is selected as the motionvalue;

(8) a video coding device as defined in item (1), characterized in thatthe selection of the frame orthogonal transform mode or the fieldorthogonal transform mode is made in such a way that the frameorthogonal transform mode is selected when frame prediction mode isselected for the codable block and the field orthogonal transform modeis selected when the field prediction mode is selected for the codableblock;

(9) a video coding device as defined in any of items (3) to (8),characterized in that the motion vector estimation is carried out forall codable blocks in a frame even if a part or all of codable blocks ofthe frame are encoded in intraframe coding mode.

A video coding device according to the present invention is of the typewhich is capable of performing the interframe video coding withadaptively selecting either frame or field orthogonal transform mode byusing a prediction error value of the frame predictive mode, aprediction error value of the field predictive mode and motion vectors,and which is featured by the followings:

(1) The prediction error values calculated respectively in the frameprediction mode and in the field prediction mode are compared with eachother to determine which one is larger than the other and the orthogonaltransform mode is selected as follows: The field orthogonal transformmode is selected when the following inequality (4) as to specifiedfunctions f(x) and g(y) is established:

    f(x)>g(y)                                                  (4)

where x is a prediction error value in frame prediction mode and y is aprediction error value in field prediction mode. The greater than sign(>) in the inequality (4) may be replaced by the greater than/equal sign(≧). If the inequality (4) is not established, the frame orthogonaltransform mode is selected.

The above-mentioned judgment is based on such a known fact that:

with the prediction error value of the frame prediction mode beingsmaller than the prediction error value in the field prediction mode,the frame image has more highly correlated adjacent lines than the fieldimage has and the orthogonal transform is preferably made for each frameto attain the power concentration to low frequency, whereas with theprediction error value of the field prediction mode smaller than that ofthe frame prediction mode, the field image has more highly correlatedadjacent lines than the frame image has and the orthogonal transform ispreferably made for each field to attain the power concentration to lowfrequency.

(2) A motion value is calculated from a motion vector obtained by motionprediction and it is compared with a specified value. The orthogonaltransform mode is selected according to whether the calculated value islarger than the specified value or not. With the motion value beinglarger than the specified, the interfield motion is large and thecorrelation between the adjacent lines is reduced. If the orthogonaltransform is now performed for a unit frame, unwanted verticalhigh-frequency components may be easily produced. Therefore, under theabove-mentioned condition, the orthogonal transform is preferablyconducted for a unit field. On the other hand, when the motion value issmall, the interfield motion value is considered to be small. In thiscase, the correlation between adjacent lines in each block is high andthe orthogonal transform may be effectively made for a unit frame inview of attaining the power concentration to a low frequency range. Onthe above-mentioned reason, the frame orthogonal transform mode isselected when the motion value is smaller than the specified value,whereas the field orthogonal transform mode is selected when the motionvalue is larger than the specified value.

(3) The orthogonal transform mode is decided according to the selectionof the frame or field prediction mode for the motion prediction. Namely,the frame orthogonal transform mode is applied when the frame predictionmode is selected, whereas the field orthogonal mode is applied when thefield prediction mode is selected.

(4) The motion prediction is performed irrespective of theintraframe-coded frame and interframe-prediction coded frame.Information obtained by the motion prediction is applied to any one ofmethods to select either the frame or field orthogonal transform mode.

Referring now to the accompanying drawings, preferred embodiments of thepresent invention will be described in detail as follows:

FIG. 6 is a block diagram for explaining a video coding device embodyingthe present invention, which comprises an input terminal 41, ansubtracter 42, intraframe-interframe coding mode selecting portion 43, achange-over switch 44, a frame-to-field converting portion 45, anorthogonal transform mode selecting portion 46, a change-over switch 47,an orthogonal transform coder 48, a multiplexor (MUX) 49, an outputterminal 50, an orthogonal transform decoder 51, a field-to-frameconverter 52, a change-over switch 53, an adder 54, a change-over switch55, a frame memory 56, a predictor 57 and a motion estimating portion58.

The coding procedure is the same as the conventional example of FIG. 1.A frame to be coded is divided into macro-blocks of 2N×2N pixels whichare inputted into the device through the input terminal 41. Eachmacro-block denoted by X has an interlaced structure. Namely, itsodd-numbered lines correspond to pixels of odd-numbered fields andeven-numbered lines correspond to pixels of even-numbered fields.

By using the macro-block X and a reference frame stored in the framememory 56, the motion estimating portion 58 determines a motion vectorMV and, at the same time, decides which mode of prediction to be madeper frame or field and produces a prediction mode selecting signal PC.The prediction to be made per frame is called frame prediction mode andthe prediction per field is called field prediction mode. The number ofmotion vectors is one per macro-block for frame prediction mode and twoper macro-block (one vector for an odd-numbered field and one vector foran even-numbered field) for field prediction mode. For bidirectionalprediction referring to preceding and succeeding reference frames, thenumber of vectors is two times the above-mentioned corresponding number.

The predictor 57 calculates a predicted macro-block P by using a motionvector MV according to a prediction mode selecting signal PC. Thesubtracter 42 determines a difference between macro-blocks X and P toobtain a interframe differential macro-block E. Theintraframe-interframe coding-mode selecting portion 43 compares themacro-block X with the macro-block E, decides which macro-block X or Eis to be encoded and generates a coding-mode selecting signal EC tooperate the change-over switch 44. The intraframe coding mode is appliedwhen X is selected and the interframe mode is applied when E isselected. The macro-block selected by the change-over switch 44 isdesignated by Fr. This macro-block Fr has an interlaced structure.

The macro-block Fr is transferred to the field converter 45 wherein itis subjected to frame-to-field conversion. The converted macro-block isdenoted by Fi. The upper half of the macro-block Fi is composed of 2N×Npixels of the odd-numbered fields and the lower half is composed of 2N×Npixels of the even-numbered fields.

An orthogonal transform-mode selecting signal TC is generated to controlchange-over switch 47 to select either one of macro-blocks Fr and Fi.The orthogonal transform coder 48 encodes the selected macro-block B of2N×2N pixels by orthogonal transformation in 4 blocks (upper left, upperright, lower left and a lower right), each having N×N pixels. The frameorthogonal transform coding mode is applied when Fr is selected and thefield orthogonal transform coding mode is applied when Fi is selected.

An output Y of the orthogonal transform coder 48 is multiplexed with themode signals TC, EC and PC and a motion vector MV by the multiplexor 49which generates, at its output terminal 50, a multiplexed output Z to betransmitted or recorded. To obtain an image reproducible at a decodingside, a coding side performs the decoding operation simultaneously withthe coding operation and a decoded macro-block is stored in the framememory 56. The encoded data Y enters into the orthogonal transformdecoder 51 which decodes the data by orthogonal transformation andobtains a decoded macro-block Fr'.

The field-to-frame converter 52 converts the macro-block Fr' into framearrangement by reversing the procedure of the frame-to-field conversion.The converted macro-block is denoted by Fi'. Either of the macro-blocksFr' and Fi' is selected by the change-over switch 53 according to theorthogonal transform mode selecting signal TC. The macro-block Fr' isselected for the frame orthogonal transform mode and the macro-block Fi'is selected for the field orthogonal transform mode.

The adder 54 adds a selected macro-block X' to a predictive macro-blockP to obtain a resulting macro-block E'. Either of the macro-blocks X'and E' is selected by the change-over switch 55 according to theintraframe-interframe coding mode selecting signal EC. The macro-blockX' is selected for the intraframe coding mode and the macro-block E' isselected for the interframe coding mode. A decoded macro-block D' isthus obtained. On completion of processing one frame, a completelydecoded frame is obtained, which will be used as a reference frame forcoding a succeeding frame.

The embodiment of FIG. 6 differs from the conventional device of FIG. 1by the orthogonal transform mode selecting portion 46 of the FIG. 6which receives input information generated by the motion estimatingportion 58 while the orthogonal transform mode selecting portion 6 ofFIG. 1 receives macro-blocks Fr and F1.

In the conventional device, the motion estimating portion 18 does notwork for an intraframe-coded frame whose macro-blocks are all to beencoded in the intraframe coding mode. Furthermore, in the intraframecoding mode, information generated by the motion estimating portion isnot used for coding even when treating with any other frame than theintraframe-coded frame. On the contrary, in the embodiment of thepresent invention, the motion estimating portion 58 in theabove-mentioned cases operates to generate information for selecting theorthogonal transform mode. Of course, this may never bring any increasein the hardware since the existing portion is used.

It is also possible to fix the orthogonal transform mode in theintraframe coding mode to the frame orthogonal transform mode or thefield orthogonal transform mode (although the embodiment is not sodesigned). In this case, the intraframe-coded frame does not require theoperation of the motion estimating portion 58. This may save the powerconsumption of the device but may not realize the adaptive applicationof the orthogonal transform mode, resulting in decreasing codingefficiency and reproducible image quality. The present embodiment canapply the suitable orthogonal transform to each macro-block in theintraframe mode according to the estimated motion value to improve thecoding characteristic.

FIG. 7 is a block diagram showing an example of the orthogonal transformmode selecting portion shown in FIG. 6. In FIG. 7, numeral 61 designatesa comparator. In this example, prediction error values calculated by themotion estimating portion 58 are used for judgment of selecting theframe orthogonal transform mode or the field orthogonal transform mode.In this case, the comparator 61 receives a prediction error value EFr ofthe frame prediction and a prediction error value EFi of the fieldprediction and compares them. An orthogonal transform mode changing-oversignal TC is generated according to the comparison result. Theprediction error value of the field prediction is a total of aprediction error value for an odd-numbered field and a prediction errorvalue for an even-numbered field.

As is apparent from FIGS. 6 and 7, the embodiment of the presentinvention eliminates the need for adders and related components shown inFIG. 5 for the conventional device. Namely, it may have a considerablysaved hardware. Furthermore, in comparison with the conventional deviceusing a pixel-value of a macro-block for judgment, the presentembodiment uses a prediction error value only, thereby reducing thequantity of data to be processed for judgment. The necessary processingtime is correspondingly shortened. FIG. 8 shows an output characteristicof the comparator of FIG. 7. The characteristic curve is obtained whenthe equation (4) has the following expressions (5):

    f(x)=x

    g(y)=y                                                     (5)

A judgment is made according to the category whereto the predictionerror value EFr of the frame prediction and the prediction value EFi ofthe field prediction belong when they are plotted on the diagram of FIG.8. When plotted points lying on a straight line on the diagram, theselection of the frame or field prediction mode depends on theexpression (4) having a greater than sign (>) or a greater than/equalsign (≧). In the present embodiment, the mode selection judgment is madeusing only the inequality relation between the prediction error valuesEFr and EFi. Of course, many other kinds of functions can be used asf(x) and g(y).

FIG. 9 shows another example of the orthogonal transform mode selectingportion shown in FIG. 6. In FIG. 9, numeral 62 designates a motion valuecalculating portion and numeral 63 designates a comparator.

In this embodiment, a motion vector calculated by the motion estimatingportion of FIG. 6 is used for judgment of the orthogonal transform modeselection. The motion value calculating portion 62 receives a motionvector MV and calculates a motion value of a macro-block therefrom. Thecomparator 63 receives the calculated motion value and compares it witha specified constant. The orthogonal transform mode is determinedaccording to the comparison result. Namely, the frame orthogonaltransform mode is selected if the motion value is smaller than theconstant. If not so, the field orthogonal transform mode is selected.

FIG. 10 shows an example of the motion value calculating portion 62shown in FIG. 9. In FIG. 10, numeral 71 designates a change-over switch,72 a motion value calculator, 73 a maximal value selecting unit and 74 acontrol unit.

Horizontal and vertical components of motion vectors to be inputted intothe motion value calculating portion 62 are designated respectively by(MV1x, MV1y), (MV2x, MV2y) and so on. A sequence of MV1 (=(MV1x, MV1y)),MV2 (=(MV2x, MV2y)) and so on corresponds to a sequence of all motionvectors of, e.g., the frame prediction or the field prediction foreven-numbered fields, which are obtained by the motion estimatingportion 58 shown in FIG. 6.

The control unit 74 operates the change-over switch 71 to enter themotion vectors (MV1, MV2 and so on) one by one into the motion valuecalculator. A horizontal component of each vector is indicated by MVxand a vertical component is indicated by MVy. The motion valuecalculator 72 subsequently calculates motion values corresponding torespective motion vectors. The maximal value selecting unit 73 comparesthe subsequently calculated motion values with each other and determinesa maximal value among them, which is used as a motion value of themacro-block.

FIG. 11 shows another example of the motion value calculating portionshown in FIG. 9. In FIG. 11, numeral 75 designates an adder, 76 aregister and 77 a latch. Other components similar in function to thoseof FIG. 10 are indicated by the same reference numbers.

All motion vectors are inputted one by one into the motion valuecalculator 72. The adder 75 and the register 76 constitutes anaccumulator. The total sum of the motion values of the motion vectors isdefined as a motion value of the macro-block. The latch 77 is controlledso as to hold the total sum. On completion of fixing the total sum atthe latch 77, the register 76 is reset for succeeding accumulatingoperation.

FIG. 12 shows an example of the motion value calculator shown in FIGS.10 and 11. In FIG. 12, numerals 81a and 81b designate absolute value(ABS) calculators and numeral 82 designates an adder.

The ABS calculators 81a and 81b calculate an absolute value |MVx| of ahorizontal component MVx and an absolute value |MVy| of a verticalcomponent MVy, and the adder 82 calculates a sum of the absolute values.A calculation result |MVx|+|MVy| is outputted as a motion value of amacro-block. The ABS calculation with the sign inversion may be done byinverting the all bits and adding 1 if each vector component isexpressed as a complement of 2.

FIG. 13 shows another example of the motion value calculator shown inFIGS. 10 and 11. In FIG. 13, there are shown latches 83a and 83b, achange-over switch 84, an adder 85, a register 86, a latch 87 and acontrol unit 88.

This calculator is similar to the calculator of FIG. 12 in using|MVx|+|MVy| as a motion value, but it may commonly use an ABS calculator81 for MVx and MVy by time-difference processing to save the size of thehardware. The reason the present embodiment can conduct thetime-difference processing is as follows: In comparison with the priorart device that has to process a large number of pixel values of amacro-block for selecting the orthogonal transform mode, the presentembodiment may process a considerably reduced number of motion vectorsfor the same purpose, producing a time allowance for inputting the datainto the orthogonal transform mode selecting portion. The reduction ofthe quantity of data necessary for judgment may reduce the number ofoperations and save the processing time.

FIGS. 14(a) to 14(g) are illustrative of a timing chart of controlsignal and data in the embodiment of FIG. 13. A horizontal component MVxand a vertical component MVy of a motion vector inputted into the motionvalue calculator 72 are held by the latches 83a and 83b respectively. Inthis case, the latches receive data when inverted OE1 shown in FIG.14(a) is shifted from low to high. The data held by the latches areinputted into the ABS calculator 81 by the action of the change-overswitch at a specified time interval. An output of the ABS calculator 81is accumulatively added by the adder 85 and the register 86. The latch87 is controlled to hold a value of |MVx|+|MVy| only. The control unitcontrols the above-mentioned sequential operations.

When the portions shown in FIGS. 11 and 13 are combined, the adder 85,register 86 and latch 87 of the FIG. 13 are integrated respectively withthe adder 75, register 76 and latch 77 of FIG. 11. By doing so, thehardware of the device according to the present invention can be furthersaved in size.

FIG. 15 shows another example of the motion value calculator shown inFIGS. 10 and 11. In FIG. 15, numerals 91a and 91b designate multipliers.Other portions similar in function to those shown in FIG. 12 are giventhe same reference numerals.

In the present embodiment, a horizontal component MVx and a verticalcomponent MVy of each motion vector are squared respectively by thecorresponding multipliers 91a and 91b and a total of the squared values,i.e., MVx² +MVy² is outputted as a motion value of a macro-block.

FIG. 16 shows a further example of the motion value calculator shown inFIGS. 10 and 11. In FIG. 16, numeral 91 designates a multiplier andother portions similar in function to those of FIG. 12 are given thesame references.

The operation of this calculator is the same as described for thecalculator of FIG. 13. It uses the same motion value MVx² +MVy² asdescribed in the embodiment of FIG. 15. A multiplier is also usedcommonly for determining MVx and MVy at a specified time interval inorder to save the scale of the hardware.

FIG. 17 is a block diagram showing another embodiment of a video codingdevice according to the present invention, where the portions are shownwith the same reference numbers as like portions of FIG. 6. The codingprocedure is just the same as described for the embodiment of FIG. 6.The present embodiment may omit the orthogonal transform mode control 46and the orthogonal transform mode signal TC, which are shown in FIG. 6,and use a prediction mode signal, in place of the signal TC, to controlchange-over switches 47 and 53. Namely, the frame orthogonal transformmode is selected when the signal PC indicates the frame prediction modeand the field orthogonal transform mode is selected when the signal PCindicates the field prediction mode. Consequently, in comparison withthe conventional device, this embodiment of FIG. 17 has a considerablyreduced in size of hardware by omitting the orthogonal transform modeselecting portion 6 of the prior art device.

As is apparent from the foregoing, the present invention has thefollowing advantages:

(1) In comparison with the conventional device that uses a pixel valueof each macro-block as data for judgment on selecting the orthogonaltransform mode, the video coding device according to the presentinvention uses motion prediction error values or motion vectors for thesame purpose and, thereby, may process a reduced quantity of data withresults of considerably saving a processing time necessary for selectingthe orthogonal transform mode and of considerably reducing the size ofthe necessary hardware.

(2) The transform mode selecting portion is composed of a comparator.Namely, there is no need for the multiplier and the adder used in theprior art device. This may considerably reduce the size of the necessaryhardware.

(3) By using motion vectors for judgment of the orthogonal transformmode to be selected it becomes possible to reduce a quantity ofmultipliers, adders and subtracters in comparison with the conventionaldevice, thereby attaining a considerably reduction in its hardware size.

(4) By directly applying the result of selecting the frame or fieldprediction mode to the selection of the frame or field orthogonaltransform mode, the device according to the present invention maycompletely omit the transform mode selecting portion, which is used inthe prior art device, thereby realizing further reduction of itshardware size.

(5) For a intraframe-coded frame which macro-blocks are all to be codedin intraframe coding mode, the present invention uses the motionestimating portion for selecting the orthogonal transform mode, whilethe same portion is at standstill in the prior art device. This makes itpossible to use a motion information of the intraframe-coded frame toselect a suitable one of the frame orthogonal transform mode and thefield orthogonal transform mode for each macro-block of theintraframe-coded frame. Furthermore, the above-mentioned processing isperformed by the existing motion estimating portion, i.e., there is noneed of increasing the size of its hardware. In comparison with theprior art device that uses the fixed frame or field orthogonal transformmode for the intraframe-coded frame, the present inventive device thatselectively apply suitable one of the frame orthogonal transform modeand the field orthogonal transform mode may have a signal-to-noise ratioimproved by 0.3 db frame on an average by simulation. The considerableimprovement of the image quality has been recognized by the subjectiveevaluation. The above-mentioned simulation results were obtained byapplying the standard coding method called "test model" used forstandardizing according to the international video coding standardMPEG-2 to 60 frames of a test image "cheer-leader." The effect of theimage quality improvement according to the present invention is not onlylimited to the intraframe-coded frame but also spreads to all otherinterframe-coded frame to be predicted from the intraframe-coded frame.

(6) As described above, the present inventive device can eliminate mostprocessing operations necessary for selecting the orthogonal transformmode, which has required, heretofore, a large hardware, by using themotion information from the motion estimating portion for selecting theframe or field orthogonal transform mode. This feature has a good andconsiderable effect.

We claim:
 1. A video coding device capable of dividing each frame of avideo sequence obtained by interlaced scanning into codable blocks andgenerating an image block predicted from a preceding-frame and asucceeding frame by using motion vectors to be determined by estimation,said estimation being performed in one of two selectable modescomprising a frame prediction mode to predict by using a motion vectorper codable block and a field prediction mode to predict by using amotion vector per odd-numbered field comprising odd-numbered lines andby using a motion vector per even-numbered field comprisingeven-numbered lines, said device including a motion estimation circuitfor determining an error value of frame mode prediction and an errorvalue of field mode prediction and an orthogonal transform modeselecting circuit for adaptively selecting a field orthogonal transformmode when the error value of the frame-mode prediction is larger thanthe error value of the field mode prediction and selecting a frameorthogonal transform mode when the error value of the frame modeprediction is smaller than an error value of the field mode predictionand wherein the orthogonal transform mode selecting circuit isresponsive only to the error value of the frame mode prediction and theerror value of the field mode prediction produced by said motionestimating circuit.
 2. A video coding device as in claim 1 wherein theerror value of the field mode prediction is the total error value forboth the odd-numbered and even-numbered fields.
 3. A video coding deviceas defined in claim 1, characterized in that the orthogonal transformmode selecting circuit has comparing means to compare an error value ofthe frame mode prediction with an error value of the field modeprediction.
 4. A video coding device as in claim 3 wherein the comparingmeans has an output characteristic of f(x)≧g(y) where x is the errorvalue in the flame-mode prediction and y is the error value in the fieldmode prediction.
 5. A video coding device as in claim 4 wherein f(x)=xand f(y)=y.